2 |
Applying the Transformer to Character-level Transduction ...
|
|
|
|
Abstract:
The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP tasks. Yet for character-level transduction tasks, e.g. morphological inflection generation and historical text normalization, there are few works that outperform recurrent models using the transformer. In an empirical study, we uncover that, in contrast to recurrent sequence-to-sequence models, the batch size plays a crucial role in the performance of the transformer on character-level tasks, and we show that with a large enough batch size, the transformer does indeed outperform recurrent models. We also introduce a simple technique to handle feature-guided character-level transduction that further improves performance. With these insights, we achieve state-of-the-art performance on morphological inflection and historical text normalization. We also show that the transformer outperforms a strong baseline on two other character-level transduction tasks: grapheme-to-phoneme ... : Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume ...
|
|
URL: https://dx.doi.org/10.3929/ethz-b-000518998 http://hdl.handle.net/20.500.11850/518998
|
|
BASE
|
|
Hide details
|
|
3 |
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Applying the Transformer to Character-level Transduction
|
|
|
|
In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume (2021)
|
|
BASE
|
|
Show details
|
|
5 |
Do Explicit Alignments Robustly Improve Multilingual Encoders? ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
The Paradigm Discovery Problem
|
|
|
|
In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (2020)
|
|
BASE
|
|
Show details
|
|
10 |
Emerging Cross-lingual Structure in Pretrained Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|